Introduction
Get a Quick Summary
It’s always a good idea to glance at the data
Grouping and Summarizing Operations
A very common operation is to do things by group(s) then create new summary variables.
Grouping and Summarizing Operations
Do multiple operations at once
Mapping functions to groups
We’ll use the map function to map the sum function to each element in the list
Grouping and Summarizing Operations
We can even do this with modeling and other operations.
This extracts the coefficients from a model run for each group.
Visualization
Visualizing data requires that you: - Consider carefully the information you want to display - And then how you want to display it
Tell a story with the data.
And have some fun with it!
ggplot2
The most widely used visualization package in R
Layers
Visualization can be thought of in a layered fashion
Start with the base, then build up
More pipes
ggplot2 uses a + as a pipe to add layers
We can pipe to any ggplot as we did before (%>%)
Aesthetics
Aesthetics (aes) map variables to visual properties
Geoms are the geometric units we want to display
Stats
We can also use ggplot2 to create statistics we want to visualize
Typically used indirectly when geoms are called
Can be used for more direct control
Scales
Scales are used to add specifications to axes, colors, etc.
model_variables %>%
filter(award_total_amount >= 1e6) %>%
ggplot() +
geom_density(aes(x = award_total_amount,
color = gender,
fill = gender),
alpha = .2) +
scale_x_continuous(breaks = c(1e6, 5e6, 1e7, 2.5e7, 5e7),
trans = 'log') +
scale_fill_viridis_d(begin = .25, end = .5) +
scale_color_viridis_d(begin = .25, end = .5)
Facets
Facets allow another dimension to plots by group
- facet_grid: returns a matrix of like dimensions
- facet_wrap: more flexible specification
Themes
Themes allow for customization
Two uses of a - a built-in versions (e.g. theme_minimal) - DIY (theme(…))
For the theme function, each argument, takes on a specific value or an element function:
element_rect
element_line
element_text
element_blank
Interactivity
Interactivity is a must-have tool for web-based presentation
Use to enhance exploration of the data - Not just because one can
Allows for additional dimensions
Even useful for exploring raw data
Interactivity
General
- plotly
- also used in Python, Matlab, Julia
- can convert
ggplot2 images to interactive ones
- highcharter
- general wrapper for highcharts.js
- works with some R packages out of the box
- rbokeh
- like plotly, it also has cross language support
Interactivity
Specific functionality:
Plotly
traces - add_, work similar to geoms
modes - allow for points, lines, text and combinations
aesthetics - variables are denoted with ~, constants do not use - x =~ var1 vs x = 2
Plotly
Plotly uses the standard pipe %>%
Plotly
Use ggplotly to turn our formerly static plots into interactive ones.
Python examples
Grouping and Summarizing data
Visualization
matplotlib is the most common visualization module in Python, though it’s fairly dated at this point. As such we’ll use a ggplot implementation in python called plotnine.
Unfortunately for plotly, the interactivity makes it unusable within the R notebook (at present), so you may need to switch to Anaconda or other IDE to try other modules like plotly. Even Python users will still use R for easier visualization though, so feel free to do what you like there, then use ggplot etc. in R when the time comes.
That said, I’ll show a couple plots
LS0tCnRpdGxlOiAiTW9kdWxlIDI6IFN1bW1hcml6aW5nIGFuZCBWaXN1YWxpemluZyBEYXRhIgpvdXRwdXQ6IAogIGh0bWxfbm90ZWJvb2s6IAogICAgaGlnaGxpZ2h0OiBweWdtZW50cwogICAgdGhlbWU6IHNhbmRzdG9uZQogICAgY3NzOiBvdGhlci5jc3MKZWRpdG9yX29wdGlvbnM6IAogIGNodW5rX291dHB1dF90eXBlOiBpbmxpbmUKLS0tCgpgYGB7ciBpbml0LCBlY2hvPUZBTFNFfQojIHRoZXNlIG9wdGlvbnMgYXJlIHByaW1hcnkgdXNlZnVsIHRvIHRoZSBjcmVhdGlvbiBvZiB0aGUgaHRtbCBkb2N1bWVudAprbml0cjo6b3B0c19jaHVuayRzZXQoCiAgZWNobz1ULCAKICBldmFsID0gRiwKICBtZXNzYWdlID0gRiwgCiAgd2FybmluZyA9IEYsIAogIGNvbW1lbnQgPSBOQSwKICBSLm9wdGlvbnM9bGlzdCh3aWR0aD0xMjApLCAKICBjYWNoZS5yZWJ1aWxkPUYsIAogIGNhY2hlPUYsCiAgZmlnLmFsaWduPSdjZW50ZXInLCAKICBmaWcuYXNwID0gLjcsCiAgZGV2ID0gJ3N2ZycsIAogIGRldi5hcmdzPWxpc3QoYmcgPSAndHJhbnNwYXJlbnQnKQopCmBgYAoKYGBge3IgY2F0Y2h1cCwgZXZhbD1UUlVFfQpsaWJyYXJ5KHRpZHl2ZXJzZSkKZGVtb2dyYXBoaWNzID0gcmVhZC5jc3YoJ2RhdGEvZGVtb3NfYW5vbnltaXplZC5jc3YnKQppZHMgPSByZWFkLmNzdignZGF0YS9pZHNfYW5vbnltaXplZC5jc3YnKQptb2RlbF92YXJpYWJsZXMgPSByZWFkLmNzdignZGF0YS9tb2RlbF92YXJpYWJsZXNfYW5vbnltaXplZC5jc3YnKQpgYGAKCiMjIEludHJvZHVjdGlvbgoKIyMgR2V0IGEgUXVpY2sgU3VtbWFyeQoKSXQncyBhbHdheXMgYSBnb29kIGlkZWEgdG8gZ2xhbmNlIGF0IHRoZSBkYXRhCgpgYGB7ciBzdW1tYXJ5fQpnbGltcHNlKGRlbW9ncmFwaGljcykKYGBgCmBgYHtyfQpzdW1tYXJ5KGRlbW9ncmFwaGljc1ssMToxMF0pCmBgYAoKCgojIyBHcm91cGluZyBhbmQgU3VtbWFyaXppbmcgT3BlcmF0aW9ucwoKQSB2ZXJ5IGNvbW1vbiBvcGVyYXRpb24gaXMgdG8gZG8gdGhpbmdzIGJ5IGdyb3VwKHMpIHRoZW4gY3JlYXRlIG5ldyBzdW1tYXJ5IHZhcmlhYmxlcy4KCmBgYHtyIGdyb3VwX2J5LCBldmFsPVRSVUV9CmRlbW9ncmFwaGljcyAlPiUgCiAgZ3JvdXBfYnkobGlidXNlcikgJT4lIAogIHN1bW1hcmlzZShhZ2UgPSBtZWFuKGFnZSwgbmEucm0gPSBUKSkgCmBgYAoKCgojIyBHcm91cGluZyBhbmQgU3VtbWFyaXppbmcgT3BlcmF0aW9ucwoKRG8gbXVsdGlwbGUgb3BlcmF0aW9ucyBhdCBvbmNlCgpgYGB7ciBncm91cF9ieTIsIGV2YWw9VFJVRX0KZGVtb2dyYXBoaWNzICU+JSAKICBncm91cF9ieShsaWJ1c2VyKSAlPiUgCiAgc3VtbWFyaXNlKGFnZV9tZWFuID0gbWVhbihhZ2UsIG5hLnJtID0gVCksCiAgICAgICAgICAgIGFnZV9zZCA9IHNkKGFnZSwgbmEucm0gPSBUKSwKICAgICAgICAgICAgYWdlX21heCA9IG1heChhZ2UsIG5hLnJtID0gVCksCiAgICAgICAgICAgIHByb3BfbWFsZSA9IG1lYW4oZ2VuZGVyPT0nTWFsZScsIG5hLnJtID0gVCkpIApgYGAKCgoKIyMgIE1hcHBpbmcgZnVuY3Rpb25zIHRvIGdyb3VwcwoKV2XigJlsbCB1c2UgdGhlIGBtYXBgIGZ1bmN0aW9uIHRvICptYXAqIHRoZSBgc3VtYCBmdW5jdGlvbiB0byBlYWNoIGVsZW1lbnQgaW4gdGhlIGxpc3QKCmBgYHtyIG1hcCwgZXZhbD1UUlVFfQp4ID0gbGlzdCgxOjMsIDQ6NiwgNzo5KQptYXAoeCwgc3VtKQpgYGAKCgoKIyMgR3JvdXBpbmcgYW5kIFN1bW1hcml6aW5nIE9wZXJhdGlvbnMKCldlIGNhbiBldmVuIGRvIHRoaXMgd2l0aCBtb2RlbGluZyBhbmQgb3RoZXIgb3BlcmF0aW9ucy4gCgpUaGlzIGV4dHJhY3RzIHRoZSBjb2VmZmljaWVudHMgZnJvbSBhIG1vZGVsIHJ1biBmb3IgZWFjaCBncm91cC4KCmBgYHtyIGdyb3VwX2J5MywgZXZhbD1UUlVFfQpkZW1vZ3JhcGhpY3MgJT4lIAogIGRyb3BfbmEocmFjZSkgJT4lIAogIGdyb3VwX3NwbGl0KHJhY2UpICU+JSAgIAogIG1hcCh+bG0oYXdhcmRfdG90YWxfYW1vdW50IH4gZ2VuZGVyLCBkYXRhID0gLikpCmBgYAoKCiMjIFZpc3VhbGl6YXRpb24KClZpc3VhbGl6aW5nIGRhdGEgcmVxdWlyZXMgdGhhdCB5b3U6Ci0gQ29uc2lkZXIgY2FyZWZ1bGx5IHRoZSBpbmZvcm1hdGlvbiB5b3Ugd2FudCB0byBkaXNwbGF5Ci0gQW5kIHRoZW4gaG93IHlvdSB3YW50IHRvIGRpc3BsYXkgaXQKClRlbGwgYSBzdG9yeSB3aXRoIHRoZSBkYXRhLgoKQW5kIGhhdmUgc29tZSBmdW4gd2l0aCBpdCEKCgojIyBnZ3Bsb3QyCgpUaGUgbW9zdCB3aWRlbHkgdXNlZCB2aXN1YWxpemF0aW9uIHBhY2thZ2UgaW4gUgoKCgojIyBMYXllcnMKClZpc3VhbGl6YXRpb24gY2FuIGJlIHRob3VnaHQgb2YgaW4gYSBsYXllcmVkIGZhc2hpb24KClN0YXJ0IHdpdGggdGhlIGJhc2UsIHRoZW4gYnVpbGQgdXAKCiMjIE1vcmUgcGlwZXMKCmBnZ3Bsb3QyYCB1c2VzIGEgYCtgIGFzIGEgcGlwZSB0byBhZGQgbGF5ZXJzCgpgYGB7ciBnZ3Bsb3QyLCBldmFsPUZBTFNFfQpsaWJyYXJ5KGdncGxvdDIpCmdncGxvdChkYXRhKSArCiAgZ2VvbV9wb2ludChhZXMoeCA9IHZhcjEsIHkgPSB2YXIyKSkgKwogIGdlb21fbGluZSgpICsKICB0aGVtZShwbG90LmNhcHRpb24gPSBlbGVtZW50X3RleHQoc2l6ZSA9IDYpKQpgYGAKCldlIGNhbiBwaXBlICp0byogYW55IGdncGxvdCBhcyB3ZSBkaWQgYmVmb3JlIChgICU+JSBgKQoKCgojIyBBZXN0aGV0aWNzCgoqKkFlc3RoZXRpY3MqKiAoYGFlc2ApIG1hcCB2YXJpYWJsZXMgdG8gdmlzdWFsIHByb3BlcnRpZXMKCioqR2VvbXMqKiBhcmUgdGhlIGdlb21ldHJpYyB1bml0cyB3ZSB3YW50IHRvIGRpc3BsYXkKCgojIyBBZXN0aGV0aWNzCgpgYGB7ciBhZXN0aGV0aWNzfQptb2RlbF92YXJpYWJsZXMgJT4lIAogIGZpbHRlcihhd2FyZF90b3RhbF9hbW91bnQgPiAxZTcpICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9kZW5zaXR5KGFlcyh4ID0gYXdhcmRfdG90YWxfYW1vdW50LCAKICAgICAgICAgICAgICAgICAgIGNvbG9yID0gZmFjdG9yKGdlbmRlciksCiAgICAgICAgICAgICAgICAgICBmaWxsID0gZmFjdG9yKGdlbmRlcikpLAogICAgICAgICAgICAgICBhbHBoYSA9IC4yKSArIAogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSAoMToxMCkgKiAxZTcsIHRyYW5zID0gJ2xvZycpCmBgYAoKYGBge3IgYWVzdGhldGljczJ9CmRlbW9ncmFwaGljcyAlPiUgCiAgZmlsdGVyKGF3YXJkX3llYXJfc3RhcnQgPCAyMDIwICYgYXdhcmRfeWVhcl9zdGFydCA+IDE5OTApICU+JSAgCiAgZ2dwbG90KGFlcyhhd2FyZF95ZWFyX3N0YXJ0LCBhd2FyZF90b3RhbF9sb2cpKSArIAogIGdlb21fc21vb3RoKGFlcyhjb2xvcj1mYWN0b3IobGlidXNlcikpKQpgYGAKCgojIyBTdGF0cwoKV2UgY2FuIGFsc28gdXNlIGdncGxvdDIgdG8gY3JlYXRlIHN0YXRpc3RpY3Mgd2Ugd2FudCB0byB2aXN1YWxpemUKClR5cGljYWxseSB1c2VkIGluZGlyZWN0bHkgd2hlbiBnZW9tcyBhcmUgY2FsbGVkCgpDYW4gYmUgdXNlZCBmb3IgbW9yZSBkaXJlY3QgY29udHJvbAoKYGBge3IgZ2dzdGF0czJ9CmdncGxvdChtb2RlbF92YXJpYWJsZXMsIGFlcyhhZ2UsIGF3YXJkX3RvdGFsX2Ftb3VudCkpICsKICBnZW9tX3BvaW50KGFscGhhID0gLjAyKSArCiAgc3RhdF9lbGxpcHNlKGNvbG9yID0gJyNmZjU1MDAnKQpgYGAKCgoKIyMgU2NhbGVzCgpTY2FsZXMgYXJlIHVzZWQgdG8gYWRkIHNwZWNpZmljYXRpb25zIHRvIGF4ZXMsIGNvbG9ycywgZXRjLgoKYGBge3Igc2NhbGVzfQptb2RlbF92YXJpYWJsZXMgJT4lIAogIGZpbHRlcihhd2FyZF90b3RhbF9hbW91bnQgPj0gMWU2KSAlPiUgCiAgZ2dwbG90KCkgKwogIGdlb21fZGVuc2l0eShhZXMoeCA9IGF3YXJkX3RvdGFsX2Ftb3VudCwgCiAgICAgICAgICAgICAgICAgICBjb2xvciA9IGdlbmRlciwKICAgICAgICAgICAgICAgICAgIGZpbGwgPSBnZW5kZXIpLAogICAgICAgICAgICAgICBhbHBoYSA9IC4yKSArIAogIHNjYWxlX3hfY29udGludW91cyhicmVha3MgPSBjKDFlNiwgNWU2LCAxZTcsIDIuNWU3LCA1ZTcpLCAKICAgICAgICAgICAgICAgICAgICAgdHJhbnMgPSAnbG9nJykgKwogIHNjYWxlX2ZpbGxfdmlyaWRpc19kKGJlZ2luID0gLjI1LCBlbmQgPSAuNSkgKwogIHNjYWxlX2NvbG9yX3ZpcmlkaXNfZChiZWdpbiA9IC4yNSwgZW5kID0gLjUpIApgYGAKCgojIyBGYWNldHMKCkZhY2V0cyBhbGxvdyBhbm90aGVyIGRpbWVuc2lvbiB0byBwbG90cyBieSBncm91cAoKLSAqKmZhY2V0X2dyaWQqKjogcmV0dXJucyBhIG1hdHJpeCBvZiBsaWtlIGRpbWVuc2lvbnMKLSAqKmZhY2V0X3dyYXAqKjogbW9yZSBmbGV4aWJsZSBzcGVjaWZpY2F0aW9uCgpgYGB7ciBmYWNldHN9Cm1vZGVsX3ZhcmlhYmxlcyAlPiUgCiAgZ2dwbG90KCkgKwogIGdlb21fZGVuc2l0eShhZXMoeCA9IGF3YXJkX3RvdGFsX2Ftb3VudCwgCiAgICAgICAgICAgICAgICAgICBjb2xvciA9IGxpYnVzZXIsCiAgICAgICAgICAgICAgICAgICBmaWxsID0gbGlidXNlciksCiAgICAgICAgICAgICAgIGFscGhhID0gLjIpICsKICBmYWNldF93cmFwKH5nZW5kZXIpCmBgYAoKCgoKIyMgVGhlbWVzCgpgVGhlbWVzYCBhbGxvdyBmb3IgY3VzdG9taXphdGlvbgoKVHdvIHVzZXMgb2YgYSAKLSBhIGJ1aWx0LWluIHZlcnNpb25zIChlLmcuIGB0aGVtZV9taW5pbWFsYCkKLSBESVkgIChgdGhlbWVgKC4uLikpCgpGb3IgdGhlIHRoZW1lIGZ1bmN0aW9uLCBlYWNoIGFyZ3VtZW50LCB0YWtlcyBvbiBhIHNwZWNpZmljIHZhbHVlIG9yIGFuIGVsZW1lbnQgZnVuY3Rpb246CgotIGBlbGVtZW50X3JlY3RgCi0gYGVsZW1lbnRfbGluZWAKLSBgZWxlbWVudF90ZXh0YAotIGBlbGVtZW50X2JsYW5rYAoKCiMjIFRoZW1lcwoKYGBge3IgdGhlbWVzMX0KbW9kZWxfdmFyaWFibGVzICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zbW9vdGgoYWVzKHggPSBhZ2UsCiAgICAgICAgICAgICAgICB5ID0gYXdhcmRfdG90YWxfYW1vdW50LAogICAgICAgICAgICAgICAgY29sb3IgPSBsaWJ1c2VyKSwKICAgICAgICAgICAgICAgYWxwaGEgPSAuMikgKyAKICB0aGVtZV9taW5pbWFsKCkKYGBgCgoKYGBge3IgdGhlbWVzMn0KbW9kZWxfdmFyaWFibGVzICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9zbW9vdGgoYWVzKHggPSBhZ2UsCiAgICAgICAgICAgICAgICB5ID0gYXdhcmRfdG90YWxfYW1vdW50LAogICAgICAgICAgICAgICAgY29sb3IgPSBsaWJ1c2VyKSwKICAgICAgICAgICAgICAgYWxwaGEgPSAuMikgKyAKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChzaXplPTEyKSwKICAgICAgICBwYW5lbC5ncmlkLm1pbm9yLnggPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgcGxvdC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGNvbG9yID0gJ3Jvc3licm93bicpLAogICAgICAgIHBhbmVsLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9ICdwYXBheWF3aGlwJykpCmBgYAoKIyMgSW50ZXJhY3Rpdml0eQoKSW50ZXJhY3Rpdml0eSBpcyBhIG11c3QtaGF2ZSB0b29sIGZvciB3ZWItYmFzZWQgcHJlc2VudGF0aW9uCgpVc2UgdG8gZW5oYW5jZSBleHBsb3JhdGlvbiBvZiB0aGUgZGF0YQotIE5vdCBqdXN0IGJlY2F1c2Ugb25lIGNhbgoKQWxsb3dzIGZvciBhZGRpdGlvbmFsIGRpbWVuc2lvbnMKCkV2ZW4gdXNlZnVsIGZvciBleHBsb3JpbmcgcmF3IGRhdGEKCgoKIyMgSW50ZXJhY3Rpdml0eQoKCkdlbmVyYWwgCgotIFtwbG90bHldKGh0dHBzOi8vcGxvdC5seS9yLykKICAtIGFsc28gdXNlZCBpbiBQeXRob24sIE1hdGxhYiwgSnVsaWEKICAtIGNhbiBjb252ZXJ0IGBnZ3Bsb3QyYCBpbWFnZXMgdG8gaW50ZXJhY3RpdmUgb25lcwogIAotIFtoaWdoY2hhcnRlcl0oaHR0cDovL2prdW5zdC5jb20vaGlnaGNoYXJ0ZXIvKQogIC0gZ2VuZXJhbCB3cmFwcGVyIGZvciBoaWdoY2hhcnRzLmpzIAogIC0gd29ya3Mgd2l0aCBzb21lIFIgcGFja2FnZXMgb3V0IG9mIHRoZSBib3gKICAKLSBbcmJva2VoXShodHRwOi8vaGFmZW4uZ2l0aHViLmlvL3Jib2tlaC8pCiAgLSBsaWtlIHBsb3RseSwgaXQgYWxzbyBoYXMgY3Jvc3MgbGFuZ3VhZ2Ugc3VwcG9ydAoKCgojIyBJbnRlcmFjdGl2aXR5CgpTcGVjaWZpYyBmdW5jdGlvbmFsaXR5OgoKLSBbRFRdKGh0dHBzOi8vcnN0dWRpby5naXRodWIuaW8vRFQvKQogIC0gaW50ZXJhY3RpdmUgZGF0YSB0YWJsZXMKICAKLSBbbGVhZmxldF0oaHR0cHM6Ly9yc3R1ZGlvLmdpdGh1Yi5pby9sZWFmbGV0LykKICAgIC0gbWFwcyB3aXRoIE9wZW5TdHJlZXRNYXAKICAgIAotIFt2aXNOZXR3b3JrXShodHRwOi8vZGF0YXN0b3JtLW9wZW4uZ2l0aHViLmlvL3Zpc05ldHdvcmsvKQogICAgLSBOZXR3b3JrIHZpc3VhbGl6YXRpb24KICAgIAoKCiMjIFBsb3RseQoKYHRyYWNlc2AKLSBgYWRkX2AsIHdvcmsgc2ltaWxhciB0byBnZW9tcwogIApgbW9kZXNgCi0gYWxsb3cgZm9yIHBvaW50cywgbGluZXMsIHRleHQgYW5kIGNvbWJpbmF0aW9ucwoKYGFlc3RoZXRpY3NgCi0gdmFyaWFibGVzIGFyZSBkZW5vdGVkIHdpdGggYH5gLCBjb25zdGFudHMgZG8gbm90IHVzZQotIGB4ID1+IHZhcjFgIHZzIGB4ID0gMmAKCgoKCiMjIFBsb3RseQoKUGxvdGx5IHVzZXMgdGhlIHN0YW5kYXJkIHBpcGUgYCU+JWAKCmBgYHtyIHBsb3RseX0KbGlicmFyeShwbG90bHkpCgptb2RlbF92YXJpYWJsZXMgJT4lIAogIHBsb3RfbHkoeCA9IH5nZW5kZXIsIHkgPSB+IGFnZSkgJT4lIAogIGFkZF9ib3hwbG90KGNvbG9yID1+IGdlbmRlcikKYGBgCgoKCiMjIFBsb3RseQoKYGBge3IgcGxvdGx5MX0KbGlicmFyeShwbG90bHkpCgptb2RlbF92YXJpYWJsZXMgJT4lIAogIGdyb3VwX2J5KGxpYnVzZXIsIGdlbmRlcikgJT4lIAogIHN1bW1hcmlzZShhd2FyZCA9IG1lYW4oYXdhcmRfdG90YWxfYW1vdW50KSkgJT4lIAogIHBsb3RfbHkoeCA9IH5saWJ1c2VyLCAKICAgICAgICAgIHkgPSB+YXdhcmQsIAogICAgICAgICAgY29sb3IgPSB+Z2VuZGVyLAogICAgICAgICAgdGV4dCA9IH5yb3VuZChhd2FyZCksIAogICAgICAgICAgdGV4dHBvc2l0aW9uID0gJ2F1dG8nLCAKICAgICAgICAgIHR5cGUgPSAnYmFyJykgJT4lIAogIGxheW91dChiYXJnYXAgPSAwLjI1LCAKICAgICAgICAgYmFyZ3JvdXBnYXAgPSAwLjI1KQpgYGAKCgoKIyMgUGxvdGx5CgpgYGB7ciBwbG90bHkyfQppbml0ID0gZ2xtKGF3YXJkX3RvdGFsX2Ftb3VudCA+PSA1MDAwMDAwIH4gYWdlKmxpYnVzZXIsIAogICAgICAgICAgIGRhdGEgPSBtb2RlbF92YXJpYWJsZXMsIAogICAgICAgICAgIGZhbWlseSA9IGJpbm9taWFsKQoKbW9kZWxfdmFyaWFibGVzICU+JSAKICBtb2RlbHI6OmFkZF9wcmVkaWN0aW9ucyhpbml0LCB0eXBlID0gJ3Jlc3BvbnNlJykgJT4lIAogIHBsb3RfbHkoeCA9IH5hZ2UsIHkgPSB+IHByZWQpICU+JSAKICBhZGRfbGluZXMoY29sb3IgPX4gbGlidXNlciwgbGluZSA9IGxpc3Qoc2hhcGUgPSAic3BsaW5lIikpICU+JSAKICBsYXlvdXQodGl0bGUgPSAnUHJlZGljdGVkIFByb2IuIEF3YXJkID4gMSBtaWwnKQpgYGAKCgoKIyMgUGxvdGx5CgpVc2UgZ2dwbG90bHkgdG8gdHVybiBvdXIgZm9ybWVybHkgc3RhdGljIHBsb3RzIGludG8gaW50ZXJhY3RpdmUgb25lcy4gCgpgYGB7ciBwbG90bHkzfQpwID0gbW9kZWxfdmFyaWFibGVzICU+JSAKICBnZ3Bsb3QoKSArCiAgZ2VvbV9kZW5zaXR5KGFlcyh4ID0gbG9nKGF3YXJkX3RvdGFsX2Ftb3VudCksIAogICAgICAgICAgICAgICAgICAgY29sb3IgPSBsaWJ1c2VyLAogICAgICAgICAgICAgICAgICAgZmlsbCA9IGxpYnVzZXIpLAogICAgICAgICAgICAgICBhbHBoYSA9IC4yKSArIAogIGZhY2V0X3dyYXAofiBnZW5kZXIpIApnZ3Bsb3RseSgpCmBgYAoKCgoKIyMgUHl0aG9uIGV4YW1wbGVzCgojIyMgSW5pdAoKYGBge3B5dGhvbiBweV9pbml0LCBlbmdpbmUucGF0aD0gJy9Vc2Vycy9taWNsL2FuYWNvbmRhMy9iaW4vcHl0aG9uJ30KaW1wb3J0IHBhbmRhcyBhcyBwZAppbXBvcnQgbnVtcHkgYXMgbnAKCmRlbW9ncmFwaGljcyA9IHBkLnJlYWRfY3N2KCdkYXRhL2RlbW9zX2Fub255bWl6ZWQuY3N2JykKaWRzID0gcGQucmVhZF9jc3YoJ2RhdGEvaWRzX2Fub255bWl6ZWQuY3N2JykKbW9kZWxfdmFyaWFibGVzID0gcGQucmVhZF9jc3YoJ2RhdGEvbW9kZWxfdmFyaWFibGVzX2Fub255bWl6ZWQuY3N2JykKYGBgCgoKIyMjICBHcm91cGluZyBhbmQgU3VtbWFyaXppbmcgZGF0YQoKYGBge3B5dGhvbiBweV9zdW1tYXJ5fQpkZW1vZ3JhcGhpY3MuZGVzY3JpYmUoaW5jbHVkZSA9ICdhbGwnKQpgYGAKYGBge3B5dGhvbiBweV9zdW1tYXJ5X251bWVyaWN9CmRlbW9ncmFwaGljcy5kZXNjcmliZShpbmNsdWRlID0gW25wLm51bWJlcl0pCmBgYAoKYGBge3B5dGhvbiBweV9zdW1tYXJ5X3N0cmluZ30KZGVtb2dyYXBoaWNzLmRlc2NyaWJlKGluY2x1ZGUgPSBbbnAub2JqZWN0XSkKYGBgCgoKCgpgYGB7cHl0aG9uIHB5X2dyb3VwX2J5LCBldmFsPVRSVUV9CmxpYl9ncm91cCA9IGRlbW9ncmFwaGljcy5ncm91cGJ5KCdsaWJ1c2VyJywgc29ydD1UcnVlLCApCgojIGF1dG9tYXRpY2FsbHkgY2hvb3NlcyBudW1lcmljCmxpYl9ncm91cC5tZWFuKCkKYGBgCgoKYGBge3B5dGhvbiBweV9ncm91cF9ieTIsIGV2YWw9VFJVRX0KbGliX2dyb3VwLmdldF9ncm91cCgwKS5oZWFkKCkKYGBgCgoKYGBge3B5dGhvbiBweV9ncm91cF9ieTMsIGV2YWw9VFJVRX0KbGliX2dyb3VwLnNpemUoKQpgYGAKCmBgYHtweXRob24gcHlfZ3JvdXBfYnk0LCBldmFsPVRSVUV9CmxpYl9ncm91cC5kZXNjcmliZSgpCmBgYAojIyMgTWFwcGluZyBhIGZ1bmN0aW9uCgpgYGB7cHl0aG9uIHB5X21hcH0KeCA9IFtbMSwyLDNdLCBbNCw1LDZdLCBbNyw4LDldXQpsaXN0KG1hcChucC5zdW0sIHgpKQpgYGAKCmBgYHtweXRob24gcHlfbWFwMn0KZGVtb2dyYXBoaWNzLnNlbGVjdF9kdHlwZXMoJ251bWJlcicpLmFwcGx5KG5wLm1lYW4pCmBgYAoKCiMjIyBWaXN1YWxpemF0aW9uCgpgbWF0cGxvdGxpYmAgaXMgdGhlIG1vc3QgY29tbW9uIHZpc3VhbGl6YXRpb24gbW9kdWxlIGluIFB5dGhvbiwgdGhvdWdoIGl0J3MgZmFpcmx5IGRhdGVkIGF0IHRoaXMgcG9pbnQuIEFzIHN1Y2ggd2UnbGwgdXNlIGEgZ2dwbG90IGltcGxlbWVudGF0aW9uIGluIHB5dGhvbiBjYWxsZWQgYHBsb3RuaW5lYC4KClVuZm9ydHVuYXRlbHkgZm9yIHBsb3RseSwgdGhlIGludGVyYWN0aXZpdHkgbWFrZXMgaXQgdW51c2FibGUgd2l0aGluIHRoZSBSIG5vdGVib29rIChhdCBwcmVzZW50KSwgc28geW91IG1heSBuZWVkIHRvIHN3aXRjaCB0byBBbmFjb25kYSBvciBvdGhlciBJREUgdG8gdHJ5IG90aGVyIG1vZHVsZXMgbGlrZSBgcGxvdGx5YC4gIEV2ZW4gUHl0aG9uIHVzZXJzIHdpbGwgc3RpbGwgdXNlIFIgZm9yIGVhc2llciB2aXN1YWxpemF0aW9uIHRob3VnaCwgc28gZmVlbCBmcmVlIHRvIGRvIHdoYXQgeW91IGxpa2UgdGhlcmUsIHRoZW4gdXNlIGdncGxvdCBldGMuIGluIFIgd2hlbiB0aGUgdGltZSBjb21lcy4KClRoYXQgc2FpZCwgSSdsbCBzaG93IGEgY291cGxlIHBsb3RzCgpgYGB7cHl0aG9uIHBsb3RuaW5lfQppbXBvcnQgcGxvdG5pbmUKYGBgCgpgYGB7cHl0aG9uIGdnMX0KZHBsb3QgPSBkZW1vZ3JhcGhpY3NbKGRlbW9ncmFwaGljcy5hd2FyZF95ZWFyX3N0YXJ0IDwgMjAyMCkgJiAoZGVtb2dyYXBoaWNzLmF3YXJkX3llYXJfc3RhcnQgPiAxOTkwKSAmIChkZW1vZ3JhcGhpY3MuYXdhcmRfdG90YWxfbG9nID4gMTEpXQpkcGxvdApnZ3Bsb3QoZHBsb3QsIGFlcyh4PSdhd2FyZF95ZWFyX3N0YXJ0JywgeT0nYXdhcmRfdG90YWxfbG9nJykpICsgZ2VvbV9zbW9vdGgoYWVzKGdyb3VwID0gJ2xpYnVzZXInLCBjb2xvciA9ICdsaWJ1c2VyJykpCmBgYAoKCmBgYHtweXRob24gZ2cyfQp5ZWFyX2F3YXJkX2F2ZXJhZ2UgPSBkZW1vZ3JhcGhpY3NbZGVtb2dyYXBoaWNzLmxpYnVzZXIgPT0gMV0uZ3JvdXBieShbJ2F3YXJkX2hvbWVfZGVwdCcsICdhd2FyZF95ZWFyX3N0YXJ0J10pLm1lYW4oKQp5ZWFyX2F3YXJkX2F2ZXJhZ2UKYGBgCgoKCmBgYHtweXRob24gcHlfcGxvdGx5X2luaXR9CmltcG9ydCBwbG90bHkKaW1wb3J0IHBsb3RseS5wbG90bHkgYXMgcHkKZnJvbSBwbG90bHkub2ZmbGluZSBpbXBvcnQgaW5pdF9ub3RlYm9va19tb2RlCmltcG9ydCBwbG90bHkuZ3JhcGhfb2JqcyBhcyBnbwpwbG90bHkub2ZmbGluZS5pbml0X25vdGVib29rX21vZGUoY29ubmVjdGVkPVRydWUpCmBgYAoKCmBgYHtweXRob24gcHlfcGxvdGx5fQp5MCA9IG5wLnJhbmRvbS5yYW5kbig1MCktMQp5MSA9IG5wLnJhbmRvbS5yYW5kbig1MCkrMQoKdHJhY2UwID0gZ28uQm94KAogICAgeT15MAopCnRyYWNlMSA9IGdvLkJveCgKICAgIHk9eTEKKQpkYXRhID0gW3RyYWNlMCwgdHJhY2UxXQpweS5pcGxvdChkYXRhKQpgYGAKCgpgYGB7cHl0aG9ufQpgYGAK